Method and System for Aggregating Reviews and Searching within Reviews for a Product

ABSTRACT

The embodiments disclosed herein include new, more efficient ways to collect product reviews from the Internet, aggregate reviews for the same product, and provide an aggregated review to end users in a searchable format. One aspect of the invention is a graphical user interface on a computer that includes a plurality of portions of reviews for a product and a search input area for entering search terms to search for reviews of the product that contain the search terms.

RELATED APPLICATIONS

This application is a continuation of U.S. Application No. 13/102,023,filed May 5, 2011, entitled “Method and System for Aggregating Reviewsand Searching within Reviews for a Product,” which is incorporatedherein by reference in its entirety.

This application is related to U.S. patent application Ser. No.11/012,500, filed Dec. 14, 2004, entitled “Method and System for Findingand Aggregating Reviews for a Product,” which application isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosed embodiments relate generally to search engines. Moreparticularly, the disclosed embodiments relate to methods, systems, andgraphical user interfaces for finding, aggregating, and providingreviews for a product.

BACKGROUND

Many Internet users research a product (e.g., a Canon digital camera) ora service (e.g., a hotel room in Hawaii) before obtaining it. Currently,the approach that many users follow is to use Internet search engines.Users issue a search query that contains the product or service nameplus terms like “review” and sift through the myriad of results that arereturned by the search engine. While some of these results do containreviews about the product, many results do not. Moreover, it is verydifficult to get a holistic view of all the reviews. In short, thisprocess is fairly cumbersome, time consuming, and inefficient.

Alternatively, users may go to Web sites that provide reviews andratings for products, such as www.amazon.com, www.epinions.com, andwww.circuitcity.com. A few sites, such as www.rottentomatoes.com,movies.yahoo.com, and www.consumersearch.com, aggregate reviews for thesame product from several sites. However, some of these sites aggregatereviews manually, which is very inefficient and limits the number ofreviews that can be gathered for each product. Moreover, all of thesesites are very limited in the tools they provide users to navigatethrough the reviews.

Thus, it would be highly desirable to provide tools that enable users tomore efficiently conduct research on the products and services they areinterested in obtaining (e.g., by purchase, lease, rental, or othersimilar transaction).

SUMMARY

The embodiments disclosed herein include new, more efficient ways tocollect product reviews from the Internet, aggregate reviews for thesame product, and provide an aggregated review to end users in asearchable format.

In one aspect of the invention, a server collects information containingproduct reviews for a plurality of products and automatically extractsthe product reviews from the collected information. For at least some ofthe extracted product reviews, the server identifies a particularproduct that is associated with the extracted product review. For eachparticular product in at least a subset of the plurality of products,the server generates aggregated review information for the particularproduct based on a plurality of extracted product reviews that areassociated with the particular product. The server stores the extractedproduct reviews and the aggregated review information.

In another aspect of the invention, a server collects product reviewsfor a plurality of products and automatically identifies particularproducts that are associated with particular product reviews. For eachparticular product in at least a subset of the plurality of products,the server automatically generates aggregated review information for theparticular product including frequently appearing phrases in the productreviews associated with the particular product. The server automaticallystores the product reviews and the aggregated review information.

As used in the specification and claims, the word “product” refers toboth products and services. Moreover, “product” encompasses virtuallyany product, service or combination thereof that can be bought, leased,rented, or similarly obtained. Exemplary products include, withoutlimitation, consumer products, business products, movies, books,restaurants, hotels, and travel packages.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned aspects of theinvention as well as additional aspects and embodiments thereof,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1 is a block diagram illustrating an exemplary distributed computersystem according to an embodiment of the invention.

FIG. 2A is a block diagram illustrating an aggregated reviews backendserver 102 in accordance with one embodiment of the present invention.

FIG. 2B is a block diagram illustrating an aggregated reviews frontendserver 100 in accordance with one embodiment of the present invention.

FIG. 3 is a flowchart representing a method for collecting andaggregating reviews for a product according to one embodiment.

FIG. 4 is a flowchart representing a method for displaying an aggregatedreview for a product according to one embodiment.

FIG. 5 is a flowchart representing a method for searching aggregatedreviews for a product according to one embodiment.

FIGS. 6A-6D are schematic screen shots of an exemplary graphical userinterface of a product information module (FIG. 6A) and a product reviewpresentation module (FIGS. 6B-6D) according to one embodiment.

DESCRIPTION OF EMBODIMENTS

Methods, systems, and graphical user interfaces are described that showhow to find, aggregate, and provide reviews for a product. Referencewill be made to certain embodiments of the invention, examples of whichare illustrated in the accompanying drawings. While the invention willbe described in conjunction with the embodiments, it will be understoodthat it is not intended to limit the invention to these particularembodiments alone. On the contrary, the invention is intended to coveralternatives, modifications and equivalents that are within the spiritand scope of the invention as defined by the appended claims.

Moreover, in the following description, numerous specific details areset forth to provide a thorough understanding of the present invention.However, it will be apparent to one of ordinary skill in the art thatthe invention may be practiced without these particular details. Inother instances, methods, procedures, components, and networks that arewell-known to those of ordinary skill in the art are not described indetail to avoid obscuring aspects of the present invention.

FIG. 1 is a block diagram illustrating an exemplary distributed computersystem according to an embodiment of the invention. This system includesclient computer 104, aggregated reviews backend server 102, aggregatedreviews frontend server 100, multiple Web sites such as Web sites 108and 110, and communication network(s) 106 for interconnecting thesecomponents. Client 104 includes graphical user interface (GUI) 112.Aggregated reviews backend server 102 collects product reviews frommultiple Web sites 108-110, identifies particular products that areassociated with particular product reviews, generates aggregated reviewinformation for particular products, and stores the product reviews andthe aggregated review information. Aggregated reviews frontend server100 receives and responds to requests from client 104 to provide anaggregated review for a product and/or to search within reviews for aparticular product. GUI 112 includes portions of a plurality of reviewsfor a product and a search input area for entering search terms tosearch for reviews of the product that contain the search terms.

FIG. 2A is a block diagram illustrating aggregated reviews backendserver 102 in accordance with one embodiment of the present invention.Server 102 typically includes one or more processing units (CPU's) 202,one or more network or other communications interfaces 204, memory 206,and one or more communication buses 214 for interconnecting thesecomponents. Server 102 optionally may include a user interface 208comprising a display device 210 and a keyboard 212. Memory 206 mayinclude high speed random access memory and may also includenon-volatile memory, such as one or more magnetic disk storage devices.Memory 206 may optionally include one or more storage devices remotelylocated from the CPU(s) 202. In some embodiments, the memory 206 storesthe following programs, modules and data structures, or a subset orsuperset thereof:

-   -   an operating system 216 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a communication module 218 that is used for connecting server        102 to other computers (e.g., Web sites 108 and 110) via one or        more communication interfaces 204 (wired or wireless), which in        turn connect the backend server 102 to one or more        communications networks such as the Internet, other wide area        networks, local area networks, metropolitan area networks, and        so on;    -   a crawling module 220 for crawling Web sites (e.g., 108 and        110), which includes        -   a URL queue 222 that stores URLs to be crawled by crawler            226;        -   a link parser 224 that tells crawler 226 how to extract new            links to follow from a fetched page;        -   a crawler 226 that fetches web pages, and        -   a fetched pages buffer 228 that temporarily stores web pages            fetched by crawler 226;    -   a review extraction module 230 for extracting product reviews        from the information collected by crawling module 220, which        includes a review parser 232 that extracts content pertaining to        a review (e.g., the review text, author, and date), as well as        content that identifies the product that the review is about        (e.g. the product name, model number, and/or other product        identifiers), from the fetched pages in buffer 228;    -   an extracted reviews buffer 234 that temporarily stores the        reviews extracted by review parser 232;    -   a review aggregating module 236 that identifies particular        products that are associated with particular product reviews and        generates aggregated review information for particular products;    -   an aggregated review buffer 240 that stores aggregated review        information for a product, such as the total number of reviews        for the product, an average rating for the product, a        distribution of ratings, and/or frequently appearing phrases in        the extracted product reviews associated with the product; one        or more additional types of aggregated review information may be        stored in the aggregated review buffer 240 in other embodiments;    -   a review database 242 that stores individual reviews 244 and        aggregated reviews 246;    -   a review indexer 248 that indexes the reviews (244 and 246)        stored in the review database 242 so as produce a reviews index        250; and    -   the reviews index 250, which stores the index created by review        indexer 248.

The reviews index 250 maps terms (e.g., words and phrases) to reviews.In some embodiments, the reviews index 250 also maps other values, suchas ClusterIDs (i.e., product identifiers) or review author names, to thecorresponding reviews.

FIG. 2B is a block diagram illustrating aggregated reviews frontendserver 100 in accordance with one embodiment of the present invention.The aspects of FIG. 2B that are analogous to those in FIG. 2A are notdescribed again here. The memory 276 in frontend server 100 stores thefollowing programs, modules and data structures, or a subset or supersetthereof:

-   -   an operating system 216, as described above;    -   a communication module 218, as described above;    -   a product database 252 for storing product-related information,        which includes        -   product and vendor information 254 such as product names,            models, categories, and purchasing information; and        -   a product-to-review map 256 that associates products in            database 252 with their corresponding ClusterIDs (i.e.,            product identifiers); the ClusterIDs are mapped by the            reviews index 250 to individual reviews 244 and aggregated            reviews 246;    -   reviews index 250, as described above;    -   a review search module 258 that answers search requests, which        includes        -   a “search all reviews” application 260 for searching all            reviews in reviews database 242; and        -   a “search within reviews for a product” application 262 for            searching within the set of reviews for a particular            product, to identify those reviews containing one or more            search terms; and    -   a presentation module 264 for formatting aggregated reviews and        search results for display, including        -   a reviews formatter 266 for formatting individual reviews            and aggregated review information for display; and        -   a reviews sorter 268 for sorting the reviews for a            particular product in any of the following ways, or a subset            or superset thereof: by date, review source (e.g., by            website), review quality, or review relevance to one or more            search terms.

The reviews database 242 and reviews index 250 are generated by thebackend server 102, and are copied from the backend server to thefrontend server 100 for responding to user requests for product reviewinformation.

Referring to FIGS. 2A and 2B, each of the above identified modules andapplications corresponds to a set of instructions for performing afunction described above. These modules (i.e., sets of instructions)need not be implemented as separate software programs, procedures ormodules, and thus various subsets of these modules may be combined orotherwise re-arranged in various embodiments. In some embodiments,memory 206 or 276 may store a subset of the modules and data structuresidentified above. Furthermore, memory 206 or 276 may store additionalmodules and data structures not described above.

Although FIGS. 2A and 2B show servers 100 and 102 as a number ofdiscrete items, FIGS. 2A and 2B are intended more as functionaldescriptions of the various features which may be present in servers 100and 102 rather than as a structural schematic of the embodimentsdescribed herein. In practice, and as recognized by those of ordinaryskill in the art, items shown separately could be combined and someitems could be separated. For example, some items shown separately inFIGS. 2A and 2B could be implemented on single servers and single itemscould be implemented by one or more servers. The actual number ofservers used to implement backend server 102 and frontend server 100 andhow features are allocated among them will vary from one implementationto another, and may depend in part on the amount of data traffic thatthe system must handle during peak usage periods as well as duringaverage usage periods.

Collecting Product Reviews

FIG. 3 is a flowchart representing a method for collecting andaggregating reviews for a product according to one embodiment. Theprocess shown in FIG. 3 is performed by aggregated reviews backendserver 102 (FIGS. 1 and 2A). It will be appreciated by those of ordinaryskill in the art that one or more of the acts described may be performedby hardware, software, or a combination thereof, as may be embodied inone or more computing systems.

Backend server 102 collects product reviews, or information containingproduct reviews for a plurality of products (302).

In some embodiments, collecting information comprises selectivelycrawling review-related Web sites or portions thereof and retrievinginformation containing product reviews. Selective crawling differs fromtraditional crawling in that the crawler only follows selected links onWeb pages, rather than all links.

To initiate a crawl, crawling module 220 is given a set of starting(sometimes called “seed”) URLs (Uniform Resource Locators, i.e., webaddresses) to crawl.

Crawling module 220 puts these URLs in queue 222. Crawler 226continuously fetches pages corresponding to the URLs in the queue 222until either there more no more URLs in the queue 222, or a crawltermination condition is reached. In some embodiments, link parser 224tells crawler 226 how to extract new links to follow from a fetched pageby matching regular expression patterns. In some embodiments, linkparser 224 tells crawler 226 how to extract new links to follow from afetched page by using text classification techniques. As an example ofthe former approach, Table 1 contains pseudo code for a link parser 224that selects which links to follow on an exemplary website(www.productcompany.com) for reviews of electronics products.

TABLE 1 Exemplary Pseudo Code for Crawling Module with Link Parser  Initialize crawler queue with seed page(s) from thewww.productcompany.com website; For each URL in the queue, do thefollowing {  download page from URL, and remove the URL from the queue; extract product reviews, if any, on the page, and parse and store theextracted product reviews;  search page to see if any portion of thepage matches any predefined pattern in a set of predefined patterns(contentpattern1, contentpattern2, . . . );  if the page does not matchany of the one or more predefined patterns {   processing of the page iscomplete;}  // otherwise, continue processing the page  collect alllinks on the page;  process each link as follows {   if the URL of thelink matches any predefined pattern in a set of predefined patterns(linkpattern1, linkpattern2, . . . ) OR   if the anchor text of the linkmatches any predefined pattern in a set of predefined patterns(anchorpattern1, anchorpattern2, . . . ), do the following: {   optional: remove unneeded fields from the URL of the link;    put theURL of the link on the crawler queue;}  }// Links that do not match anyof the predefined pattern(s) are not put on the queue  // The patternsused in the processing of the crawled pages may be specific to the  //website from which product review information is being extracted.

Crawler 226 outputs the crawled pages to fetched pages buffer 228. Insome embodiments, the fetched pages buffer may contain one or more filesor other data structures. For each page that is fetched, link parser 224parses the page and attempts to extract links (URLs) to follow that, iffound, are added to queue 222. As noted above, in contrast totraditional crawls, not all the links in a page are followed by thecrawler. Rather, only those links that parser 224 identifies aspotentially leading to review pages are followed. For instance, parser224 may identify such links based on the presence of terms or patternsin the URLs of the links, or based on the anchor text of the links.Anchor text is text contained in or otherwise associated with a link.

The information containing product reviews can be collected from a widevariety of sources, including web pages and forum postings. In someembodiments, the web pages comprise Web pages of review-related Websites and/or product-related Web sites.

In some embodiments, collecting information comprises receiving productreviews in a predetermined digital format, for example as data streamsfrom multiple product reviewers. The predetermined digital format of theproduct reviews may comprise key/value pairs, or data organized inpredefined data structures. In some embodiments, collecting informationcomprises selecting information from a repository of informationcollected by a web crawler.

Backend server 102 automatically extracts product reviews from thecollected information (304). In particular, review parser 232, which isanalogous to link parser 224, extracts content pertaining to a review(e.g., the review text, author, and date), as well as content thatidentifies the product that the review is about (e.g. the product name,model number, or other features), from the fetched pages in buffer 228.The extracted information is stored in extracted reviews buffer 234. Insome embodiments, extracted reviews buffer 234 contains a list ofkey/value pairs denoting the extracted product review information, suchas the review author, content, publisher, and so on.

The information extracted from a single web page may correspond tomultiple reviews. In some embodiments, each extracted review found on aweb page is encoded and stored in a respective review record in theextracted reviews buffer 234. Table 2 lists the fields of each reviewrecord in the extracted reviews buffer 234 in an exemplary embodiment.In other embodiments, each review record may have a subset of the fieldslisted in Table 2, and may also include additional fields. For example,in some embodiments, the review records might not include the ReviewTypeand CrawlDate fields. In some embodiments, the Rating field may be anumeric field, instead of a string field. Alternately, in someembodiment, each review record may include both a string Rating fieldand a normalized numeric rating field (e.g., normalized with respect toa rating scale of 0 to 1, or 0 to 5).

In Table 2, the keyword “required” indicates a required field,“repeated” indicates a field that may occur more than once in a record(and in rare cases, may have zero occurrences), and “optional” indicatesan optional field. Optional fields may be present in some reviewrecords, while not present in other review records, for example becausesome reviews may not include the optional information.

TABLE 2 Exemplary Fields for Individual Review Records in IndividualReviews Buffer and Reviews Database required int64 ReviewID // uniqueidentifier for each review repeated string ProductNames // one or moreproduct names associated with the review required int32 ReviewType //exemplary values: editorial=1, user=2 required string Rating // productrating given by the reviewer optional string Author // author of thereview optional string Title // title of the review optional stringSummary // review summary optional string Content // review content,which is the main body of the review optional string Date // Review daterequired int32 CrawlDate // crawl date required string URL // URL atwhich review was found. // Alternately, this may be the base URL forproduct // reviews at the website at which this review was foundrequired string Publisher // publisher of the review (e.g., CNET,epinions.com) // Additional Optional Fields that may be included in someembodiments: optional int64 ClusterID // ClusterID of the productoptional float QualityScore // Quality Score of the review optionalint32 Year // Model year, or other year associated with the product //(e.g., year of first publication of a book or a movie)

The data structure fields shown in Table 2, except ReviewID, ReviewType,QualityScore and ClusterID, are filled with data from the extractedreview information if information relevant to those fields are found inthe extracted review information. The ReviewID is a unique identifierassigned to each individual review by the aggregated reviews backendserver. The ReviewType indicates whether the review is an editorialreview, or a review submitted by a user. QualityScore, which provides ameasure of the quality of a review, is determined based on the review'slength and word entropy (which measures, among other things, thediversity of the words or terms used in the review). As explained below,all reviews belonging to the same product form a “cluster” and theClusterID identifies the cluster to which a review belongs.

Clustering Reviews Identifying Reviews for the Same Product

Backend server 102 automatically identifies particular products that areassociated with particular product reviews (306). For at least some ofthe extracted product reviews, backend server 102 identifies aparticular product that is associated with the extracted product review.

In some embodiments, identifying a particular product that is associatedwith the extracted product review comprises associating a unique numberor string in the extracted product review with a particular product. Insome embodiments, the unique number or string is a product model number.In some embodiments, the unique number is an ISBN number.

In some embodiments, identifying a particular product that is associatedwith the extracted product review comprises associating a brand name anda model number in the extracted product review with a particularproduct. In some embodiments, identifying a particular product that isassociated with the extracted product review comprises associating abrand name, a model number, and a product category in the extractedproduct review with a particular product.

Different web sites often use different names for the same product,which makes it difficult to automatically determine whether two reviewsactually refer to the same product. For example, different web sitesrefer to the same Canon scanner as:

-   -   Canon CanoScan 7890a002 Flatbed 7890a002    -   Canon CanoScan LiDE 30    -   Canon CanoScan LiDE 30 Scanner    -   Canon CanoScan LiDE 30 Color Scanner    -   Canon Lide 30 (7890A002)

For products with global identifiers (such as ISBN numbers for books),identifying a particular product that is associated with the extractedproduct review is simple if the product review includes the globalidentifier. In some embodiments, in the absence of such a globalidentifier in the review, extracted brand names and model numbers (inthe above example, the brand name is “canon”, and the model numbers are“lide30” and “7890a002”), as well as category information (such as thefact that the above product is a scanner) are used to identify reviewsthat refer to the same product. As used herein, reviews about the sameproduct form a “cluster” and “clustering” refers to identifying reviewsfor the same product. As described above, in some embodiments, thisinformation is stored in review records (Table 2) by review extractionmodule 230. Extracted brand names, model numbers and categoryinformation may all be stored in the ProductNames fields of the reviewrecords. Alternately, in some embodiments each review record may includea Model field for storing a model number, a Category field for storingproduct category information, and a Names field for storing brand nameinformation for a respective product.

Sometimes, one or more of these three pieces of information (Model,Category and Brand Name) are not explicitly identified on a web page. Insuch cases, this information is deduced from the title string or otherfields of the extracted review. For example, product category and modelnumber can be inferred by the review parser 232 or the reviewaggregating module 236 as follows. Given a title string, the part of thetitle string beyond words like “for”, “with”, and “featuring” isignored. Words in the title string are traversed from right to left andthe first word (or pair of words) that is included in a data filecontaining a predefined list of product categories is considered theproduct category. In some embodiments, this data file also identifies,for each distinct product category, a list of other product categorynames for the product category that are mapped to one canonical name.For example, “notebooks”, “notebook”, “laptop” or “laptops” may all bemapped to the canonical name, “notebooks”. Thus, a title string such as“Canon canoscan lide 30 color scanner with free extras” would be mappedto the category “scanner”.

To extract model numbers, the title string is traversed from left toright and the first word that contains digits (i.e., characters from‘0’-‘9’) is concatenated with the preceding word to form a model number.In some embodiments, the resulting model number is discarded if itmatches any of the regular expression patterns corresponding toblacklisted model numbers or contains a brand name or parts of a brandname. An example of a blacklisted pattern is “pentium \d+”+” (i.e.,“pentium” followed by a string of digits). For the title string in theprevious example, “lide 30” is extracted as the model number. Note thatmore than one model number can be inferred for a product name.

In some embodiments, the brand names are identified by matching words inthe product title with known (i.e., predefined) brand names in a list ordictionary of such brand names. In some embodiments, the brand name of aproduct may be extracted from the URL of the page on which the productreview is found.

In some embodiments, two product reviews are assigned to the samecluster (product) if they have the same exact brand name, and agree ontheir model numbers and categories. Two reviews are said to agree on amodel number if the set of model numbers extracted for them have anon-empty intersection. Two categories are said to agree if they are thesame or at least one of them is unknown.

In some embodiments, two product reviews are assigned to the samecluster (product) if they have the same exact brand name and agree ontheir model numbers.

In some embodiments, reviews corresponding to products for which brandor model information cannot be determined are put into a singletoncluster of their own and are not clustered with any other reviews.

In some embodiments, when a product review is assigned to a cluster,review aggregating module 236 fills in the ClusterID field in the reviewrecord for the extracted review.

Generating Aggregate Information

For each particular product in at least a subset of the plurality ofproducts, backend server 102 generates aggregated review information forthe particular product based on a plurality of extracted product reviewsthat are associated with the particular product (308). In someembodiments, review aggregating module 236 uses the reviews for aparticular product to generate aggregated review information for theproduct that is stored in aggregated review buffer 240.

In some embodiments, the aggregated review information for theparticular product comprises a total number of reviews for the product,an average rating for the product, a distribution of the ratings for theproduct, and/or frequently appearing phrases in the extracted productreviews associated with the product. One or more additional types ofaggregated review information may be included in the aggregated reviewinformation in other embodiments.

In some embodiments, the average rating is the average of weightedratings from different web sites. In some embodiments, the weight of asingle review on a web site with K reviews for the same product is 1/K.In some embodiments, the weight given to reviews from a particular website or reviewer can be personalized by a user. For example, a user canspecify via a preferences dialog box one or more web sites (e.g.,www.consumerreports.org) and/or reviewers (e.g., Roger Ebert) preferredby the user. The ratings given in the reviews associated with the user'spreferred sources are then given higher weight when computing an averagerating for a product. In one embodiment, the ratings given in reviewsassociated with the user's preferred sources are given N times thedefault weight assigned to those reviews, where N is a value such as 2,2.5, 3 or any other appropriate value greater than 1. An exemplaryrepresentation of the average rating computation is:

${{average}\mspace{14mu} {rating}} = \frac{\sum{a_{i}{rating}_{i}}}{\sum a_{i}}$

where a_(i) represents the weight given to each rating, rating_(i), ofeach respective review i.

In some embodiments, frequently appearing phrases in the extractedproduct reviews associated with a particular product are computed byreview aggregation module 236 as follows. For all the phrases (e.g.,unigrams, bigrams or trigrams that do not span across sentences) thatoccur in reviews associated with a product (cluster), the number oftheir weighted occurrences is counted. Occurrences in the review titleor summary are given a higher weight. When computing or selectingfrequently appearing phrases, all stop words and words that occur in thetitle of the product name are ignored. Each phrase is given a scoreequal to the length of the phrase times the square root of the sum ofweighted occurrences of the phrase. A predetermined number of phraseswith the highest scores (e.g., the top 6, 7, 8, 9 or 10 phrases) that donot overlap each other are chosen as the frequent phrases and stored inaggregated review buffer 240.

Table 3 lists the fields of each aggregated review record, produced byaggregating information from all the extracted reviews for a particularproduct, and stored in the aggregated reviews buffer 240. In otherembodiments, each aggregated review record may have a subset of thefields listed in Table 3, and may also include additional fields. Aseparate instance of the aggregated review record is included in theaggregated reviews buffer 240 and in the reviews database 242 for eachdistinct product. In this example, each aggregated review recordincludes a ClusterID (identifying the product associated with thisrecord), a list of ReviewID's (each of which is uniquely associated witha review of the product), a list of the words or phrases most frequentlyfound in the reviews of the product, and a computed average rating ofall the reviews for the product. In some embodiments, an aggregatedreview record also includes one or more of the following additionalfields: a string identifying the product, a number specifying the numberof reviews in the cluster; and data concerning the distribution ofratings in the reviews for a product, such as the number of reviews withlow, medium and high ratings, as well as the number of reviews having noproduct rating.

TABLE 3 Exemplary Fields for Aggregated Review Records in AggregatedReviews Buffer and Reviews Database required int64 ClusterID // this isthe ClusterID for a particular product repeated int64 ReviewID // thisrepresents a list of ReviewID's, which are the // IDs of the individualreviews for the product repeated string // This is a list of phrasesthat FrequentPhrases frequently occur // in the reviews of the productrequired float AvgRating // This is the average rating for the product// Additional Optional Fields that may be included in some embodiments:optional int32 ReviewNum // number of reviews in cluster repeated string// name(s) of Product to which ProductName this cluster belongs repeatedint32 // score distribution for: unrated, ScoreDistribution negative,neutral, // and positive reviews

Backend server 102 automatically stores the extracted product reviews244 and the aggregated review information 246 in reviews database 242(310). In some embodiments, each stored product review in the reviewsdatabase 242 comprises a record that includes the same information asthe corresponding review record (Table 2) stored in the extractedreviews buffer 234, while in other embodiments, the stored productreview may comprise a subset of the information stored in thecorresponding review record.

As mentioned earlier, reviews indexer 248 indexes the individual productreviews 244 and the aggregated review information for a product 246 andstores the results in reviews index 250.

Displaying an Aggregated Review

Client 104 sends a request for an aggregated review for a product (402,FIG. 4). In some embodiments, the request is sent in response to auser-initiated activity, such as a user clicking on a hyperlink. Forexample, referring to FIG. 6A, a user looking at web page 650 withvarious products for sale could activate hyperlink 652 to see anaggregated review for product 654.

Referring to FIG. 4, frontend server 100 receives the request (404) foran aggregated review for a product and obtains (406) the aggregatedreview. In some embodiments, product information 254 in product database252 is linked to aggregated reviews for the corresponding products byproduct-to-review map 256. In some embodiments, map 256 containsmappings of product identifiers used by a first website or service(e.g., a product information website) to the ClusterIDs used by thefrontend server 100. The process of assigning product identifiers toproducts in the first website or service may be analogous to the processdescribed above for identifying particular products that are associatedwith particular product reviews. In some embodiments, map 256 isdetermined by matching a global identifier for a particular product(e.g., an ISBN number) in product database 252 with a correspondingreview cluster. Product-to-review map 256 is used to identify theaggregated review (in the reviews database 242) corresponding to aproduct. Presentation module 264 formats the aggregated review. In someembodiments, the reviews index 250 is used to identify the individualreviews corresponding to the aggregated review for a product. Morespecifically, the reviews index 250 maps the ClusterID of the aggregatedreview to the individual reviews for the product associated with theaggregated review.

Frontend server 100 sends (408) the aggregated review to client computer104 via communications network 106. Client 104 receives (410) anddisplays (412) the aggregated review 600 (FIG. 6B).

As shown in FIG. 6B, aggregated review 600 includes portions of reviews626 for the product and a search input area 618 for entering searchterms to search for reviews of the product that contain the searchterms. Search input area 618 is used to enter search terms for a searchthat requests only results that are within the reviews for the product.In some embodiments, aggregated review 600 includes a histogram 608 ofreview ratings for the product, an aggregate rating 604 for the product,and/or the number 606 of reviews for the product. In some embodiments,aggregated review 600 is displayed in a single Web page.

Searching

In some embodiments, review search module 258 supports two differenttypes of queries: (1) search within the reviews 262 belonging to asingle product (e.g. a search for “customer service” in the reviews fora given product) and (2) search all reviews 260 to find a particularproduct (e.g. a search for “good digital camera”). Aggregated review600, FIG. 6B, shows a search input area 618 for the first type ofsearch.

For the first type of search, a variety of user initiated activities canstart the search process, including the user entering search terms insearch input area 618, selecting a hyperlink for a frequently appearingphrase 620, selecting a hyperlink to a review source 622, or selecting ahyperlinked portion (e.g., 610, 612, 614, or 616) of histogram 608.

In response to this user-initiated activity, client 104 sends (502, FIG.5) a request to search within a plurality of reviews for a particularproduct.

Referring to FIG. 5, frontend server 100 receives (504) the searchrequest from client 104 and obtains (506) information satisfying therequest. In some embodiments, the information is obtained by reviewsearch module 258 sending two queries to reviews database 242. In thefirst query, all individual reviews (sometimes called “documents”) thatcontain the search terms are found. When searching within reviews for asingle product, the first query is restricted to the main body (e.g.,the “review content” fields) of the reviews index 250, and only toindividual reviews that belong to the product (cluster). For example,the query “customer service” for reviews in cluster 1234 would betranslated to the following:

-   -   query restricts: ClusterID=1234, body:(customer service).

A second query is then issued to obtain portions of all reviews that areto be displayed. The ordering preferences submitted by the user (e.g.,by clicking on 636, 638, 640, and/or 642) as well as the start positionare used to determine which reviews to display. When the number ofreviews for a product exceeds the maximum number of review portions tobe displayed on a single web page, the start position indicates theordinal number within an ordered set of reviews at which to beginformatting reviews for display.

Frontend server 100 sends (508, FIG. 5) at least a portion of theobtained information to client 104.

Client 104 receives (510) information satisfying the request anddisplays (512) the received information, e.g. in web page 660, FIG. 6C.In some embodiments, Client 104 displays at least portions 662 ofreviews of the product matching the search terms 664.

The information sent by frontend server 100 and received by client 104includes at least portions 662 of reviews for the particular productthat satisfy the search request. In some embodiments, this informationincludes the identities 666 of the review sources contributing to thesent information.

For the second type of search, the information is also obtained byreview search module 258 sending two queries to reviews database 242. Inthe first query, all individual reviews that contain the search termsare found, without restricting the search to a particular product(cluster). In some embodiments, the results of the first query aregrouped by cluster (product). A second query is then issued to obtainportions of at most three (or some other small number) reviews in eachcluster, which are restricted to be from different publishers. In someembodiments, information about each of the clusters, such as the totalnumber of reviews and the average rating, is also requested, obtained,and sent for display.

Graphical User Interface

The graphical user interface 600 (FIG. 6B) on client computer 104comprises a plurality of portions 626 of reviews for a product and asearch input area 618 for entering search terms to search for reviews ofthe product that contain the search terms. In some embodiments, theplurality of portions of reviews can be grouped by review source 636,sorted by date 642, and sorted by relevance to the search input 640, orsorted by quality of review when the user has not provided a searchinput. In some embodiments, more or fewer sorting and grouping optionsmay be provided for sorting and/or grouping the reviews.

In some embodiments, graphical user interface 600 also includes, eitheralone or in various combinations: an aggregate rating 604 for theproduct; a list of suggested search terms 620; a list of sources 622 forthe reviews that link back to the source Web sites or to thecorresponding reviews; at least one user selectable link for sorting theportions of reviews by date 642 or relevance to the search input 640,and/or for grouping the portions of reviews by source 636; and/or ahistogram 608 of reviews for the product. In some embodiments, clickingon a region (e.g., 610, 612, 614, or 616) in the histogram automaticallygenerates portions of at least some of the reviews corresponding to thehistogram region. For example, clicking on region 616 may generate a newdisplay showing portions of reviews having high ratings for the product.

An aggregated rating 604 is given to the product by calculating anaverage over all reviews for the product. The ratings in the individualreviews 244 and in the aggregated reviews 246 are normalized, e.g., to a0-1 or 0-5 scale. In some embodiments, after a search within reviews fora product 262, the aggregated rating 604 and number 606 of reviews arechanged to reflect the average rating and number of reviews matching thesearch. This change in the aggregated rating can help users answerquestions like, “How do reviewers that discuss ‘customer service’ ratethis product?” In some other embodiments, the aggregated rating 604 andnumber 606 of reviews are not changed to reflect the average rating andnumber of reviews matching the search.

Commonly used phrases (e.g., unigrams, bigrams or trigrams) appearing inthe reviews for the product are displayed as a list of suggested searchterms 620. Phrases that contain the product name and stop-words (such as“the”) are removed to improve the quality of the suggested search terms.Clicking on one of the phrases results in displaying just the reviewsthat contain the user selected phrase. This is equivalent to enteringthe phrase into the search box 618 with quotes around the phrase andthen initiating a search. Showing these phrases gives the user a goodidea which aspects of the product are most frequently discussed in theproduct reviews, and thus which aspects of the product might beimportant to know about.

The list of review sources 622 includes links to the sites from whichreviews for the product have been obtained, or a subset thereof. In someembodiments, after a search within reviews for a product, the list ofreview sources 622 just includes links to the sites from which reviewsmatching the search have been obtained, or a subset thereof. In someembodiments, the links lead to the main pages for the given product oneach source web site. In some embodiments, the links lead to the productreviews matching the search.

A histogram 608 with a rating distribution of the reviews includes thenumber of reviews that are unrated 610 (in some embodiments, displayedas a gray area), negative 612 (in some embodiments, displayed as a redarea), neutral 614 (in some embodiments, displayed as a yellow area) andpositive 616 (in some embodiments, displayed as a green area). In someembodiments, the rating distribution is displayed as a flat histogram,where the length of each colored bar is proportional to the number ofreviews in the category. Clicking on a histogram results in a search 262that just shows the corresponding reviews, e.g. clicking on the red bar612 results in showing just the negative reviews.

Search input area 618 allows a user to enter arbitrary search terms forthe search within reviews for a given product. As shown in FIG. 6C, insome embodiments the search terms are highlighted in portions of reviews626 that match the search query.

User selectable links permit the displayed list of reviews to be sortedin various ways, such as by:

-   -   Relevance 640: In some embodiments, editorial reviews are listed        before user reviews. In some embodiments, the reviews are sorted        by their length and information content, e.g., sorted by        Review-per-word-entropy*sqrt(Review-Length). Other scoring        functions, such as scoring reviews by an “information retrieval”        score, can also be used.    -   Date 642: The reviews are sorted by date, with the newest        reviews coming first. Reviews for which a date is unknown may be        listed at the end.    -   Review Source 636: Reviews are grouped by review site, as        represented by screen shot 670 in FIG. 6D, with at most N        reviews (e.g., 3, 4 or 5, or some other small number) shown for        each review site. At the top of each group, a link 672 to the        corresponding review site is provided, which may be the same as        the link provided to the site in the list of review sources 622.    -   Ungrouped 638 (e.g., not grouped by review source). This view is        similar to traditional search result displays. Reviews are        listed (i.e., portions of the reviews are displayed) one-by-one        in a selected order (e.g., by date or relevance) and reviews        from different sources are mixed (i.e., without grouping the        portions of reviews by source).    -   Location (not shown in aggregated review 600, FIG. 6B). This        view permits reviews to be sorted by location (e.g., for reviews        of restaurants, local businesses, movies or other products near        a particular location, such as by address, city, region, or        mailing or zip code).

In some embodiments, for reviews listed in an aggregated review (e.g.,graphical user interface 600 or 670) or reviews listed as results for asearch within reviews for a product (e.g., graphical user interface 660,FIG. 6C) the following information is displayed: review title 624, aportion 626 of the review (e.g., a snippet), the source of the review628, the date of the review 634, the type of review 630 (e.g., whetherit is an editorial user contributed review), and the name of thereviewer 632. The portion or snippet 626 of each review may be generatedusing the same or similar techniques to those used to generate snippetsof search results in an Internet search engine such as the Google searchengine. For example, when a graphical user interface is presentingreviews for a product, the portion of each review shown may be a portionmatching one or more search terms that were used to select the reviewsto be displayed. Alternately, the portion of each review shown may be abeginning portion of the review. In yet another embodiment, the portionof each review shown may be a portion determined by an automatic processto be a summary or most important portion of the review.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A method comprising: at a client having one or more processors andmemory storing one or more programs for execution by the one or moreprocessors: sending a request for an aggregated review of a product to aremote server, receiving the aggregated review of the product from theremote server, wherein the aggregated review of the product is derivedfrom a plurality of reviews of the product, displaying the aggregatedreview of the product, wherein the aggregated review of the productincludes: portions of multiple reviews of the product, wherein arespective review of the product provides a subjective evaluation of theproduct by a human, and a list of phrases associated with the product,detecting selection of a phrase in the list of phrases; and in responseto detecting selection of the phrase in the list of phrases, displaying,in the aggregated review of the product, just portions of reviews of theproduct from reviews that contain the selected phrase.
 2. The method ofclaim 1, wherein the aggregated review of the product includes a firstaverage rating for the product, and the method further includesdisplaying a second average rating of reviews matching the selectedphrase.
 3. The method of claim 1, wherein the aggregated review of theproduct includes a first number of reviews of the product, and themethod further includes displaying a second number of reviews matchingthe selected phrase.
 4. The method of claim 1, wherein the aggregatedreview of the product includes a first list of review sources that linkback to source web sites or to reviews corresponding to the product, andthe method further includes displaying a second list of review sourcesthat link back to source web sites or to reviews matching the selectedphrase.
 5. The method of claim 1, wherein the aggregated review of theproduct includes a first rating distribution of reviews for the product,and the method further includes displaying a second rating distributionfor the product from reviews matching the selected phrase.
 6. The methodof claim 5, wherein each of the first and second rating distributions ofreviews for the product includes a histogram of reviews for the product.7. The method of claim 5, wherein each of the first and second ratingdistributions of reviews for the product includes multiple ratingregions, each rating region having a respective rating for the productand being associated with a respective set of reviews of the producthaving the respective rating for the product, and the method furtherincludes: detecting selection of a respective rating region having apredefined rating for the product; and in response to detectingselection of the respective rating region, displaying, in the aggregatedreview of the product, just portions of a corresponding set of reviewsthat have the predefined rating for the product.
 8. A method comprising:at a server having one or more processors and memory storing one or moreprograms for execution by the one or more processors: receiving arequest for an aggregated review of a product from a client; preparingthe aggregated review of the product, wherein the aggregated review ofthe product is derived from a plurality of reviews of the product;sending the aggregated review of the product to the client for display,wherein the aggregated review of the product includes: portions ofmultiple reviews of the product, wherein a respective review of theproduct provides a subjective evaluation of the product by a human, andinformation for displaying a list of phrases related to the product,such that a user selection of a phrase in the list of phrases causes adisplay of just portions of reviews of the product from reviews thatcontain the selected phrase in the aggregated review of the product atthe client.
 9. The method of claim 8, further comprising: indexingindividual and aggregated product reviews to produce a reviews index,wherein the reviews index maps search terms to reviews; and storing thereviews index.
 10. The method of claim 8, wherein a respective phrase inthe list of phrases is automatically selected from the plurality ofreviews of the product in accordance with corresponding weightedoccurrences of the respective phrase in the plurality of reviews of theproduct.
 11. The method of claim 10, wherein a phrase that appears in atitle or a summary of a review of the product is given more weight thanthe phrase that appears elsewhere in the review of the product.
 12. Themethod of claim 8, wherein a phrase in the plurality of reviews of theproduct is given a score that is equal to a length of the phrase times asquare root of a sum of corresponding weighted occurrences of the phrasein the plurality of reviews of the product, and the phrase is one of thedisplayed list of phrases if its score is higher than a predefinedthreshold.
 13. The method of claim 8, wherein the aggregated review ofthe product includes a first aggregate rating for the product, and themethod further includes sending to the client information for displayinga second aggregate rating of reviews matching the selected phrase. 14.The method of claim 8, wherein the aggregated review of the productincludes a first number of reviews of the product, and the methodfurther includes sending to the client information for displaying asecond number of reviews matching the selected phrase.
 15. The method ofclaim 8, wherein the aggregated review of the product includes a firstlist of review sources that link back to source web sites or to reviewscorresponding to the product, and the method further includes sending tothe client information for displaying a second list of review sourcesthat link back to source web sites or to reviews matching the selectedphrase.
 16. The method of claim 8, wherein the aggregated review of theproduct includes a first rating distribution of reviews for the product,and the method further includes sending to the client information fordisplaying a second rating distribution for the product from reviewsmatching the selected phrase.
 17. The method of claim 16, wherein eachof the first and second rating distributions of reviews for the productincludes a histogram of reviews for the product.
 18. The method of claim16, wherein each of the first and second rating distributions of reviewsfor the product includes multiple rating regions, each rating regionhaving a respective rating for the product and being associated with arespective set of reviews of the product having the respective ratingfor the product, and the method further includes: detecting selection ofa respective rating region having a predefined rating for the product;and in response to detecting selection of the respective rating region,sending to the client information for displaying, in the aggregatedreview of the product, just portions of a corresponding set of reviewsthat have the predefined rating for the product.
 19. A system comprisingat least one server, the system comprising: one or more processors; andmemory; wherein said at least one server is configured to: receive arequest for an aggregated review of a product from a client; prepare theaggregated review of the product, wherein the aggregated review of theproduct is derived from a plurality of reviews of the product; send theaggregated review of the product to the client for display, wherein theaggregated review of the product includes: portions of multiple reviewsof the product, wherein a respective review of the product provides asubjective evaluation of the product by a human, and information fordisplaying a list of phrases related to the product, such that a userselection of a phrase in the list of phrases causes a display of justportions of reviews of the product from reviews that contain theselected phrase in the aggregated review of the product at the client.20. The system of claim 19, wherein a respective phrase in the list ofphrases is automatically selected from the plurality of reviews of theproduct in accordance with corresponding weighted occurrences of therespective phrase in the plurality of reviews of the product.
 21. Thesystem of claim 19, wherein the aggregated review of the productincludes a first aggregate rating for the product, and the methodfurther includes sending to the client information for displaying asecond aggregate rating of reviews matching the selected phrase.
 22. Thesystem of claim 19, wherein the aggregated review of the productincludes a first number of reviews of the product, and the methodfurther includes sending to the client information for displaying asecond number of reviews matching the selected phrase.
 23. The system ofclaim 19, wherein the aggregated review of the product includes a firstlist of review sources that link back to source web sites or to reviewscorresponding to the product, and the method further includes sending tothe client information for displaying a second list of review sourcesthat link back to source web sites or to reviews matching the selectedphrase.
 24. A non-transitory computer readable storage medium havingstored thereon data representing sequences of instructions, which whenexecuted by a server system having one or more processors and memory,cause the server system to: receive a request for an aggregated reviewof a product from a client; prepare the aggregated review of theproduct, wherein the aggregated review of the product is derived from aplurality of reviews of the product; send to the client for display ofthe aggregated review of the product to the client, wherein theaggregated review of the product includes: portions of multiple reviewsof the product, wherein a respective review of the product provides asubjective evaluation of the product by a human, and information fordisplaying a list of phrases related to the product, such that a userselection of a phrase in the list of phrases causes a display of justportions of reviews of the product from reviews that contain theselected phrase in the aggregated review of the product at the client.25. The non-transitory computer readable storage medium of claim 24,wherein a respective phrase in the list of phrases is automaticallyselected from the plurality of reviews of the product in accordance withcorresponding weighted occurrences of the respective phrase in theplurality of reviews of the product.
 26. The non-transitory computerreadable storage medium of claim 24, wherein the aggregated review ofthe product includes a first aggregate rating for the product, and themethod further includes sending to the client information for displayinga second aggregate rating of reviews matching the selected phrase. 27.The non-transitory computer readable storage medium of claim 24, whereinthe aggregated review of the product includes a first number of reviewsof the product, and the method further includes sending to the clientinformation for displaying a second number of reviews matching theselected phrase.
 28. The non-transitory computer readable storage mediumof claim 24, wherein the aggregated review of the product includes afirst list of review sources that link back to source web sites or toreviews corresponding to the product, and the method further includessending to the client information for displaying a second list of reviewsources that link back to source web sites or to reviews matching theselected phrase.
 29. The non-transitory computer readable storage mediumof claim 24, wherein the aggregated review of the product includes afirst rating distribution of reviews for the product, and the methodfurther includes sending to the client information for displaying asecond rating distribution for the product from reviews matching theselected phrase.